Work-based performance measurement and analysis of virtual heterogeneous machines

Presented here is a set of methods and tools developed to provide transportable measurements of performance in heterogeneous networks of machines operating together as a single virtual heterogeneous machine (VHM). The methods are work-based rather than time-based, and yield significant analytical information. A technique for normalizing the measure of useful work performed across a heterogeneous network is proposed and the reasons for using a normalized measure are explored. It is shown that work-based performance measures are better than time-based ones because they may be 1) taken while a task is currently executing on a machine, 2) taken without interrupting production operation of the machine network, 3) used to compare disparate tasks, and 4) used to perform second order analysis of machine network operation. This set of performance tools has been used to monitor the utilization of high-performance computing networks, provide feedback on algorithm design and determine the veracity of computing performance models.